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1 .  Introduction 

Many  algorithms  have  been  developed,  mainly  in  the  last  two 
decades,  for  the  solution  of  mathematical  programming  problems. 
However,  following  the  now  well  known  parametric  analysis  in  linear 
programming  [12,81,95,113]  and  a  fairly  comprehensive  treatment  of  the 
quadratic  parametric  problem  [ 17 , 22 ,60 J  *  until  very  recently,  there 
has  been  only  sporadic  activity  in  the  study  of  the  solution 
sensitivity  of  general  nonlinear  mathematical  programs  to 
perturbations  of  the  problem  parameters.  This  paper  gives  a  concise 
survey  of  the  state  of  the  art  in  that  area. 

As  the  next  two  examples  demonstrate,  the  solution  of  very  simple 
mathematical  programs  may  not  vary  at  all  or  may  change  drastically 
for  arbitrarily  small  perturbations  of  the  problem  parameters. 

Example  1. 

Consider  the  nonlinear  program: 

min  (Xj  “  G)^  +  (x^  +  1)^ 

S  .  t  .  X  >  x 
2-1 
X  >  -x 
2  -  1 


-1- 
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The  analytical  solution  of  this  problem  is  easily  seen  to  be: 

((l+e)/2,-(l+e)/2)  e  <  -1 

x(c)  =  (x^(e)  )  =  (0,0)  -\<z  <  1  • 

( (e-1 )/2 ,  (e-1 )/2)  e  >  1 

It  is  clear  that  x(e)  is  piecewise  linear,  continuous,  and 
differentiable  everywhere  except  for  e  =  +  1.  Tt  is  readily  shown  that 
the  optimal  value  function  of  this  problem,  f*(e)  =  f[x(e),e],  is  twice 
differentiable  everywhere  except  for  z  -  +  1 ,  where  it  is  only  once 
differentiable. 

Unfortunately,  as  the  next  example  illustrates,  the  solutions  of 
mathematical  programs  do  not  always  behave  so  nicely. 

Example  2. 

min  ex^ 

S  .t.  -1  £  Xj 

The  solution  of  this  problem  is  given  by  x*^=  -1  if  e  >  0;  x*^ can 
be  chosen  as  any  value  in  [-l,oo)  if  e  =0;  and  if  e<0,  there  is  no 
finite  solution  of  this  problem.  Thus,  as  e  varies  in  a  small 
neighborhood  of  the  origin  in  E* ,  the  solution  may  be  finite  and 
unique,  may  be  unbounded,  or  there  may  be  infinitely  many  solutions. 

It  should  be  clear  from  these  two  simple  examples  that  very  small 
perturbations  of  the  parameters  of  a  mathematical  program  can  cause  a 
wide  variety  of  results.  The  purpose  of  this  paper  is  to  summarize 
and  illustrate  the  work  that  has  been  done  to  date  in  1)  providing 
conditions  under  which  the  solutions  of  nonlinear  programs  are  locally 
well-behaved,  and  2)  estimating  solution  properties  as  a  function  of 
problem  parameters. 


-2- 


2.  Objective  Function  and  Constraint  Set  Continuity 

Some  of  the  earliest  work  in  sensitivity  analysis  for  nonlinear 
programming  was  concerned  with  the  variation  of  the  optimal  objective 
function  value  with  changes  in  a  parameter  appearing  in  the  right-hand 
side  of  the  constraints,  i.e.  involving  problems  of  the  form: 

minimize  f(x),  subject  to  g(x)  >  e, 
where  f:  En->-  E*,  and  g:  E*1-*-  E™.  The  theory  of  point-to-set  maps 
(cf.  Berge  [13])  has  been  used  for  much  of  the  analysis  of  this 
problem.  Hogan  [67]  has  provided  an  excellent  development  of  those 
properties  of  point-to-set  maps  which  are  especially  useful  in 
deriving  such  results. 

In  the  next  sub-section  we  present  several  definitions  and  properties 
relating  to  point-to-set  maps  which  are  needed  in  a  number  of  important 
results . 

2 . 1  Point-to-Set  Maps 

Given  two  sets  X  and  Y,  a  point-to-set  mapping,  <P ,  from  X  to  Y  is 
a  function  which  associates  with  every  point  in  X  a  subset  of  Y. 
Following  Berge  [13],  we  say  that  the  point-to-set  mapping  <P  is 
continuous  if  it  is  both  upper  semi -continuous  (use)  and  lower 
semi-continuous  (lsc)  in  X.  These  last  two  notions  are  established  by 
the  following  definitions. 

Definition  2.1.  Let  <t>  be  a  point-to-set  mapping  from  X  to  subsets  of  Y. 

i)  <j)  is  lsc  at  x  e  X  if,  for  each  open  set  SCy  satisfying 
o 

Sn<Kx  )  f  0,  there  exists  a  neighborhood  N  of  x  ,  N(x  ), 
o  o  o 

such  that  for  each  x  in  N(x  ),  <f>(x)OS  4  0. 

o 

ii)  <f>  is  use  at  x  e  X  if,  for  each  open  set  SCY  containing 
o 

4>(x  ),  there  exists  a  neighborhood  N  of  x  ,  N(x  ),  such 
o  o  o 

that  for  each  x  in  n(x  ),  <t>(x)CS. 

o 
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An  alternative  definition  of  semi-continuity  for  point-to-set 

maps  is  based  on  the  convergence  of  sets  in  the  range  space  Y.  Let 

{x_}  be  any  sequence  of  points  in  X  such  that  x  x ,  and  let  $  be  a 
11  n 

point-  to-set  mapping  from  X  to  subsets  of  Y. 

Definition  2.2.  i)  <t>  is  lsc  (or  open)  at  xeX  if,  for  each  ye]>(x)  , 

there  exists  a  value  m  and  a  sequence  {yn}c  Y  such 

that  y  ed)(x  )  for  each  n  >  m  and  y  -»■  y. 
n  n  _  n 

ii)  4)  is  use  (or  closed)  at  xeX  if  y  e<J>(x)  and  y  -+•  y 

n  n 

together  imply  that  ye<{>(x). 

Furthermore,  if  <J>  is  lsc  at  each  point  of  X,  then  it  is  said  to 
be  lsc  in  X;  and  if  <p  is  use  at  each  point  of  X  with  <Kx)  compact  for 
each  x,  then  <i>  is  said  to  be  use  in  X.  Using  these  definitions,  the 
following  results  are  easily  established  for  real-valued  functions 
(cf.  Berge  [13]).  Rummer  [74]  has  obtained  similar  results  and  has 
applied  them  to  the  particular  case  of  quasi-convex  programs. 

Theorem  2.3.  If  f  is  a  real-valued  use  (lsc)  function  defined  on 
X  x  Y  and  if  |  is  a  lsc  (use)  mapping  from  X  into  Y  such  that  for 
each  x,  <)>(x)  /  0,  then  the  (real-valued)  function  f*,  defined  by 
f*(x)  =  inf  [f(x,y)} 

is  use  (lsc). 

Theorem  2.4.  If  f  is  a  continuous  real-valued  function  defined  on  the 
space  Y  and  <J>  is  a  continuous  mapping  of  X  into  Y  such  that  <j>(x)  4  0. 
for  each  xe  X,  then  the  (real-valued)  function  f * ,  defined  by 

f*(x)  =  min  {f(y) ] 
ye  ( x) 

is  continuous  in  X.  Furthermore,  the  mapping  F,  defined  by 
F(x)  =  {y:ye<j>(x),  f(y)  =  f*(x)[  , 
is  a  use  mapping  of  X  into  Y. 

Conditions  which  imply  the  continuity  of  the  solution  of  a 
mathematical  program  have  been  given  by  Dantzig,  Folk-man,  and 
Shapiro  [25],  and  by  Robinson  and  Day  [102].  Letting 

-4- 
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f  be  a  mapping  from  a  metric,  space  X  to  E*  ,  with  H(e)C  X,  and  defining 
M(f:H(e))  =  j  xell(e) :  f(x)  =  inf  jf(y):  yeH(e)}}  Dantzig,  et.  al.  obtain 
conditions  on  the  variation  of  f  and  H  which  are  necessary  and  sufficient 
for  M(f:H(e))  to  vary  in  a  closed  manner.  When  H  is  defined  by  linear 
inequalities,  they  obtain  under  appropriate  conditions,  that  M(f:H(e))  is 
a  closed  function.  Under  this  same  hypothesis,  letting  M*(f:H(e))  denote 
the  mapping  M(f:h(e))  when  it  is  a  singleton,  they  obtain  conditions 
which  yield  the  continuity  of  M*(f:H(e))  as  a  function  of  the 
parameter  e. 

Theorem  2.5.  Let  H  be  a  point-to-set  mapping  from  the  metric  space 

T  to  the  set  of  subsets  of  E° ,  with  H(e)  a  closed  set  for  each  e  in 

T.  Let  T'  =  {e  in  T:  H(e)  £  ^}.  Suppose  H(e)  is  connected  for  each 

e  in  T' ,  and  for  some  e*  in  T' ,  H(e*)  is  compact.  Furthermore, 

assume  that  for  every  sequence  {e  |cT',  e  -*-£*  implies 

n  n 

1  im  H(c  )  =  H(e*).  Then,  if  feC(En),  the  mapping  ( f :H(e) ) 

n-»oo  n 

is  continuous  at  e*  if  e*  is  in  its  domain. 

If  g  is  an  affine  function  from  En  to  Em,  denote  by  H(g)  the 
set  {xe  E°:  g(x)  >^0}.  The  function  g  is  said  to  be  nondegenerate 
w  h  respect  to  the  set  RCEn  if  H(g) 0  R  has  a  nonempty  interior 
no  component  of  g  is  identically  zero.  The  continuity  of 
M(f:H(g)OR)  and  M*(f:H(g)OR)  as  functi  ons  of  g  are  given  by  the 
following  theorems. 

Theorem  2.6.  If  f  is  continuous  and  R  is  closed  and  convex,  then 
M(f:H(g)flR)  is  closed  at  every  nondegenerate  point  g. 

Theorem  2.7.  If  f  is  quasiconvex  or  H(g)flR  is  bounded,  then 
M*(f:H(g)DR)  is  continuous  at  a  nondegenerate  point  g  of  its 
domain. 


-5- 
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Robinson  and  Day  [102],  considering  a  general  constraint  set,  R(e) , 
provide  conditions  which  guarantee  the  continuity  of  the  point- 
to-set  mapping  whose  range  is  the  set  of  solutions  of  the 
mathematical  program 

minimize  f(x,e) 
s.t.  xeR(e) 
e  in  T, 

where  R(e)  represents  a  constraint  set  as  a  function  of  the 
parameter  e,  and  X  and  T  are  topological  spaces.  To  that  end, 
let  f*(e)  =  minx(f(x,e):  xeR(e)},  and  define  the  mapping  S:T-VX 
by  S(e)  =  {x:  f(x,e)_<  f*(c)] O  R(e) .  Assume  R(e)  is  continuous  on  T. 

Theorem  2. 8.  Let  X  be  locally  convex  and  let  R  ^  0  be  convex-valued 
on  T.  If  f(x,e)  =  min{u(x,e),  a(e)},  where  ueC(X  x  T)  and  is 
strictly  quasi-convex  in  x  for  each  fixed  e,  and  aeC(T) ,  then 
S  is  continuous  and  convex  valued  on  T. 


2.2  Right-Hand  Side  (RHS)  Perturbations 

In  this  section  we  will  be  concerned  with  problems  of  the  form: 


minimize  f(x),  s.t.,  g(x)  >_  e ,  P^e) 

where  f:  E  — »E  and  g:  En— *Em.  Associated  with  problem  P^(e)  are  the 
following  four  sets: 

i)  the  feasible  region,  R(e)  =  {x:  g(x)  >_  e}, 

ii)  the  set  B  ={e:R(e)  ^  0}, 

iii)  a  set  associated  with  the  interior  of  the  feasible  region, 
1(e)  =  (x:g(x)  >  e}, 

iv)  the  optimal  value  function,  f*(e)  =  inf  (f (x) :xcR(e) }. 


One  readily  sees  that  i) ,  iii)  and  iv)  may  each  be  viewed  as  a  point- 
to-set  mapping.  It  is  precisely  this  observation  which  has  been 
exploited  in  characterizing  the  variation  of  both  the  solution  and 
the  optimal  objective  function  value  as  functions  of  the  parameter  c. 


-6- 
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The  following  three  theorems,  due  to  Evans  and  Gould  [34], 
provide  conditions  for  the  stability  of  the  constraint  set,  i.e.  the 
feasible  region  R(e) ,  as  well  as  for  the  continuity  of  the  optimal 
objective  function.  In  the  statements  of  these  results,  we  will 
denote  by  R  a  point-to-set  mapping  from  the  set  B  C  Em  to  the  set  of 
all  subsets  of  En ,  with  the  image  of  e  in  B  given  by  R(e).  The 
interior  of  the  set  B  is  denoted  by  Int(B),  and  the  closure  of  1(e)  is 
denoted  by  1(e).  Assume  g  is  continuous. 

Theorem  2.9.  i)  The  mapping  R  is  use  at  e  if  and  only  if  there  exists  a 
vector  e'  <  e  such  that  R(e')  is  compact. 

ii)  If  R(e)  is  compact  and  1(e)  4  <b ,  then  R  is  lsc  at  e  if  and 
only  if  1(e)  =  R(e). 

* 

Theorem  2. 10.  i)  If  f  is  lsc  and  R  is  use  at  £,  then  f  is  left 
continuous  (lsc)  at  e. 

* 

ii)  If  1(e)  4  0,  f  is  use  and  R  is  lsc  at  e,  then  f  is 
right  continuous  (use)  at  e. 

Theorem  2 . 11.  Suppose  e  is  in  Int(B)  and  that  f  is  continuous,  feC(En). 
Also  assume  that  there  exists  a  vector  e'  <  e  such  that  R(e')  is  compact, 
and  that  1(e)  =  R(e).  Then  f*  is  continuous  at  e. 

Theorem  2.11  is  related  to  the  second  result  of  Berge,  i.e..  Theorem 
2.4,  in  that  Evans  and  Gould  have  given  conditions  which  imply  the 
hypotheses  of  Berge' s  theorem.  The  question  of  the  stability  of  the 
set  of  optimal  solutions  and  the  stability  of  the  optimal  objective 
function  value  have  been  addressed  by  a  number  of  authors.  Greenberg 
and  Pierskalla  [58],  referring  to  problem  P^,  have  shown  that  the  point- 
to-set  napping  (j>,  which  maps  E™  to  subsets  of  En  and  is  defined  by 
<f>  (e)  =  {xcR(e):f(x)  =  f*(t)}  is  use  at  e  if  R  is  use  at  e  and  if  f* 
is  continuous  at  e.  This  result  is  very  similar  to  one  given  by 
Dantzig,  Folkman,  and  Shapiro  [ 25]  and  by  Berge  [13].  The  essential 
differences  among  these  results  lie  in  the  use  of  semi-continuity  in 
[58]  and  the  closedness  of  maps  in  [25],  while  the  conclusion  drawn  in 
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[ 1 3  J  is  based  on  the  continuity  of  R. 

Considering  the  perturbation  function  for  convex  programming 
problems,  which  is  central  to  the  construction  of  decomposition  algo¬ 
rithms  for  large-scale  nonlinear  programs,  Hogar  [68]  established 
conditions  for  its  continuity.  That  result  has  been  extended  to  the 
case  in  which  the  parameter  of  the  problem  appears  in  the  objective 
function  as  well  as  in  the  constraints.  These  results  are  given 
in  the  next  two  theorems. 

Theorem  2.12.  Let  f*(e)  =  inf^ifCx):  xcX,  g(x)^e)  .  If 

i)  X  is  a  compact  convex  set, 

ii)  f  is  continuous  on  X, 

iii)  is  lsc  on  X,  and 

iv)  each  g.  is  strictly  concave  on  X. 

,  *  1 

tnen  f  is  continuous  on  its  domain. 

Theorem  2.13.  Let  f*(e)  =  infx(f(x,e):  xeX,  g(x,e)  ^>0}.  If 

i)  X  is  a  compact  convex  set, 

ii)  f  and  g  are  both  continuous  on  X,  and 

iii)  g  is  strictly  concave  on  X  for  each  e. 
then  f  is  continuous  on  its  domain. 

More  recently,  Clarke  [21]  has  shown  that  if  X  is  a  Banach  space 
and  f  is  locally  Lipschitz  then  programs  of  the  form  P^(e),  with  xeSCX 
are  "normal"  in  the  sense  that  generalized  Kuhn-Tucker  conditions 
can  be  shown  to  pertain,  even  in  the  absence  of  d i f ferent iabi 1 i ty  and 
convexity  assumptions.  Clarke  terms  the  program 

min  f(x) 

s.t.  g  (x)  >  e  i  =  1, . . . ,m,  „  ,  . 

n  P2(e) 

xeS,  S  closed  in  E  , 
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normal  '  if  kar ush-Kulin-Tucker  type  multipliers  exist  for  any  x  which  solvei 
?2(e)  is  said  to  be  "calm"  if  <(>(e)  is  finite  and 

lira  inf  [ <t> ( e * )  -  4>(e) }/  |e'  -  e|  >  -  oo  , 


where 

inf {f (x) :xeS,  g(x)  >_  e  }  R(e)  +  0 

4>  (e)  = 

+  00  R(e)  =  0 

Note  that  the  limit  quotient  in  (2.1)  is  a  form  of  stability  also  used 
by  Rockafellar  ( 105 ] .  Using  these  notions,  Clarke  showed  that  if  P  (e) 
is  calm  it  is  also  normal  and  if  (J>(e)  is  finite  in  a  neighborhood  of 
e  =  0,  then  P  (e)  is  calm  and  normal  for  almost  all  e  in  a  neighborhood  of 
of  0.  Conditions  sufficient  for  the  calmness  (and  hence  the  normality) 
of  P  (e)  are  given  in  the  following  theorem. 

Theore m  2.14.  I f 

i)  -g^,  i  =  l,...,m,  are  convex, 

ii)  S  is  convex  and  bounded, 

iii)  f  is  bounded  and  Lipschitz  on  S,  and 

iv)  there  exists  a  point  xeS  such  that  g(x)  >  e, 
then  P2(e)  is  calm. 


Additional  stability  results  for  right  hand  side  perturbations 
are  discussed  in  the  next  section  where  we  summarize  the  results  that 
have  been  achieved  in  the  area  of  differential  stability.  In  Section 
5,  first-  and  second-order  sensitivity  of  the  objective  function  under 
right  hand  side  perturbations  is  indicated. 
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3 .  Differential  Stability 

In  this  section  we  concentrate  on  theory  that  has  been  developed 
to  analyze  various  differentiability  properties  of  the  optimal  value 
function.  We  begin  with  a  brief  discussion  of  several  well  known 
constraint  qualifications,  one  of  which  is  often  involved  in  obtaining 
some  of  the  results. 

3 . 1  Constraint  Qualifications 

The  constraint  qualifications  used  in  mathematical  programming 
are  regularity  conditions  which  are  generally  imposed  to  insure  that 
the  set  of  Karush-Kuhn-Tucker  multipliers  corresponding  to  the  optimal 
solution  of  a  mathematical  program  is  nonempty.  We  present  here 
five  qualifications  which  are  freciuently  applied.  These  and  a  number 
of  others  are  treated  in  some  detail  by  Mangasarian  [79].  Throughout 
this  discussion,  we  shall  assume  the  constraint  set  R  =  {xeEn;  g(x)  >_  0, 
h(x)  =  0},  where  g:En  ->-E  and  h:En  -*-E^.  Define  B(x)  H  (i:g.(x)  =  0}. 

CQ1.  The  Mangasarian-Fromovitz  constraint  qualification  is  said  to 
hold  at  a  point  x*£ R  if: 

i)  there  exists  a  vector  yeEn  such  that  Vg.(x*)  y  >  0 
for  all  ieB(x*) ,  Vh (x*)  y  =  0  for  j  =  l,...,p,  and 
ii)  the  gradients  {Vh.(x*),  j  =  l,...p,}  are  linearly  independent. 

If  the  g^  are  concave  (or  even  pseudo-concave)  functions  and  the 
h j  are  affine,  then  CQ1  is  equivalent  to  the  well-known  Slater 
condition,  a  general  form  of  which  we  give  as  CQ2. 
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Definition  3.1.  If  the  function  f:  E  ►  E  is  differentiable  on  the 
convex  set  S  and  f (y )  >_  f(x)  for  all  x,  yeS,  with  Vf(x)(y-z)  >_0,  then 

f  is  said  to  be  psuedo-convex  on  S.  (The  function  f  is  said  to  be 
pseudo-concave  if  -f  is  pseudo-convex.) 

CQ2.  The  Slater  constraint  qualification  is  satisfied  at  x*eR  if  h. 
is  affine  for  each  j  ,  g_^  is  pseudo-concave  and  there  exists  a  point 
xeR  witli  g^(x)  >  0  for  eacn  i  in  B(x*). 

CQ3.  The  linear  independence  assumption  is  said  to  hold  at  x%R  if 
the  gradients  {Vg^(x*),  ieB(x*) ;  Vhj(x*),  j=l,...,p)  are  linearly 
independent . 

CQ4.  If  there  are  no  equality  constraints  and  Jfu^Vg^Cx*)  =  0  has  no 

ieB(x*) 

non-zero  solution  u.  >_  0  for  x*eR,  the  Cottle  constraint  qualification 
is  said  to  hold  at  x*.  (In  the  absence  of  equality  constraints,  CQ1 
is  equivalent  to  CQ4.) 

CQ5.  The  Kuhn-Tucker  constraint  qualification  is  satisfied  at 

x*eR  if,  for  each  non-zero  vector  zcEn  satisfying  Vg^(x*)z  >  0  for  each 

icB(x*)  and  Vh  (x*)z  =  0,  j  =  l,...,p,  z  is  tangent  to  a  once-dif- 

j 

ferentiable  arc  originating  at  x*  and  contained  in  R. 

The  relationships  that  hold  among  these  qualifications,  in 
addition  to  those  already  mentioned,  are  that  CQ3  implies  CQ1  which, 
in  turn,  is  sufficient  for  CQ5.  For  a  proof  and  further  discussion 
of  the  relationships  among  these  constraint  qualifications  see 
[9,  61]. 

Robinson  {101]  has  shown  the  equivalence  of  CQ1  and  a  form  of 
local  stability  of  the  set  of  solutions  of  a  system  of  inequalities. 
Gauvin  [49]  has  shown  that  CQ1  is  both  necessary  and  sufficient  for 
the  set  of  Lagrange  multiplier  vectors  corresponding  to  a  given  local 
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solution  of  a  general  NLP  problem  to  be  nonempty,  compact  and  convex. 

In  addition,  Gauvin  and  Tolle  [50]  establish  that  CQ1  is  preserved 
under  RHS  perturbations. 

3 . 2  Differential  Stability  of  the  Extremal  Value Function 

One  of  the  earliest  characterizations  of  the  differential  stability 
of  the  extremal  value  function  of  a  mathematical  program  was  provided 
by  Danskin  [23,24].  Addressing  the  problem  minimize  f(x,e)  subject  to 
xeR,  R  some  topological  space,  Danskin  derived  conditions  under  which 
the  directional  derivative  of  the  extremal  value  function  exists  and 
also  determined  its  representation. 

Definition  3.2.  The  (one-sided)  directional  derivative  of  the  function 
f(x)  in  the  direction  z  is  defined  to  be: 

D  f(x)  =  lim  [ (f (x+gz)  -  f(x))/3] 

8+0+ 

if  the  limit  exists. 

Theorem  3.3.  Let  R  be  non-empty  and  compact  and  let  f  and  the  partial 
derivatives  3f/3e.  be  continuous.  Then,  at  any  point  e  in  E  and  for  any 

k  1 

direction  zeE  ,  the  directional  derivative  of  f*  exists  and  is  given  by 

D  f*(e)  =  min  zv  f(x,c), 
xeS(e) 

where  S(e)  =  {x:x  minimizes  f(x,e)  over  R}. 

This  result  has  wide  applicability  in  the  sense  that  the  constraint 
space,  R,  can  be  any  compact  topological  space.  It  has  been  extended 
by  a  number  of  authors,  including  Demyanov  and  Rubinov  127],  to  other 
spaces  and  a  variety  of  functional  forms.  The  principal  restriction  of 
this  result  is  that  the  set  R  does  not  vary  with  the  parameter  e. 

However,  since  inequality  and  equality  constraints  can  be  "absorbed" 
into  the  objective  function  of  a  program  through  the  use  of  an 
appropriate  auxiliary  function  (Lagrangian,  penalty  function,  etc.), 
Danskin's  result  is  readily  applicable  to  auxiliary  function  methods. 

It  can  also  be  readily  applied  to  the  dual  of  a  convex  program  with 
right  hand  side  perturbations.  For  the  special  case  in  which  R  is 
defined  bv  inequalities,  g(x,e)  >  0,  and  f  and  g  are  convex  on  S, 
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Hogan  [66]  has  shown  that  D  t*(c)  exists  and  is  finite  for  all 

k  z 

zeE  .  The  following  theorem  presents  the  details  of  this  result  for 

f*(e)  =  inf  {f(x,e):  g(x,e)  >^0},  wiiere  X  is  a  subset  of  E°.  The 
xeX 

Lagrangian  for  this  problem  is  defined  as  L(x,u,e)  =  f(x,e)  -  ug(x,e). 

For  convenience  and  without  loss  of  generality,  we  shall  assume  that  the 
parameter  value  of  interest  is  e  =  0,  unless  otherwise  stated. 

Theorem  3.4.  Let  X  be  a  closed  and  convex  set.  Suppose  f  and  -g  are 
convex  on  X  for  each  fixed  £  and  are  continuously  differentiable  on  X  x  N(0) , 
a  neighborhood  of  e  =  0  in  E^.  If  S(0)  =  {xeX:  g(x,0)  >_  0  and  f*(0)  >_ 

f (x,0) }  is  nonempty  and  bounded,  f*(0)  is  finite,  and  there  is  a  point 

yeX  such  that  g(y,0)  >  0,  then  D  f*(0)  exists  and  is  finite  for  all 

„n  ,  z 

zeE  ,  and 


D  f*(0)=  min  max  zVeL(x,u;e)  =  min  max  {z(V  f(x,0)  -  u7  g(x,0))}, 

xeS(O)  ueK(x,0)  xeS(O)  ueK(x,0)e 

where  K(x,0)  is  the  set  of  optimal  Lagrange  mulitpliers  for  the  given 

xeS(O) . 

Some  recent  investigations  of  this  sort  have  focused  on  the  extremal 
value  function  inequality-equality  constrained  optimization  problems 
with  right  hand  side  perturbations,  of  the  form 


maximize  f(x) 


S.t.  g(x)  _>  E^ 

i  =  1 , .  .  . 

,m 

P,(e) 

h(x)  =  e 

j+m 

i  =  1 ,  •  •  • 

,P- 

J 

Let  R(e)  =  (x:  g  (x)  >  e  , 

i  1 

i  =  1... . 

,m}  ft  {x:  h 

.(x)  = 
J 

e  .  .  ,  j  =  1 . p}» 

j+m 

and  let 

1 

j  inf  { f (x) 

:  xeR(e)} 

R(e) 

¥  0 

f*(e)  = 

1 

1 

l  +  00 

R(e) 

=  0. 

We  also  define,  for  R(c) 

¥  d,  S(e) 

=  {  xe  R  ( £ ) 

:  f(x) 

=  f*(e)},  and  the 

Lagrangian  L(x,u,w,c)  H  f(x) 

-  u[g(x)-e 

]  +  w [h (x) 

-E  j  . 

Given  these 

definitions,  Gauvin  and  Tolle  [50]  have  proved  the  following  continuity 
property  of  f*(c). 


J 
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Theorem  3.5.  If  R(0)  f  0,  with  R(e)  uniformly  compact  near  e  =  0,  and 
if  CQ1  holds  for  some  x*eS(0),  then  f*(e)  is  continuous  at  e  =  0. 

FLacco  [45]  has  recently  shown  that  this  result  holds  for  the 
general  problem 

min  f (x,e) 

s.t.  g(x,e)  >_  0  P4^C^ 

h(x,e)  =  0, 

where  the  problem  functions  are  C1  in  (x,e),  R(e)  anci  g(e)  are 

defined  to  be  the  feasible  region  and  solution  sets  of  P.(e), 

£ 

respectively,  and  where  the  parameter  £  is  a  vector  in  E  . 

In  the  absence  of  equality  constraints,  Rockafellor  [103]  has  shown 
that,  under  certain  second-order  conditions,  the  function  f*(e)  of 
P3(e)  satisfies  a  stability  of  degree  two,  i.e.,  in  a  neighborhood  of 
e  =  0,  there  exists  a  twice  differentiable  function  E™-*  EP  with 
f*(e)  21  <t>(£)  and  f*(0)  =  <)>(0).  Under  this  stability  property,  bounds 
on  the  directional  derivatives  of  f*  (when  they  exist)  can  be  derived. 
For  convex  programming  problems  of  the  form  P  (e) ,  Gol'stein  [57]  has 
shown  that  a  saddle  point  condition  (summarized  subsequently)  is 
satisfied  by  the  directional  derivative  of  f*.  Gauvin  and  Tolle 
[ 50 ] »  not  assuming  convexity,  but  limiting  their  analysis  to  problem 
P  (e),  extend  the  work  of  Gol'stein  and  provide  sharp  bounds  on 
the  directional  derivative  of  f*,  also  without  requiring  the  existence 
of  second  order  derivatives.  These  results  were  extended  by  Fiacco 
and  Hutzler  [44]  to  the  more  general  problem  P4(e)  and  are  presented 
next. 

Let  L(x,u,w,e)  =  f(x,g)  -  ug(x,c)  +  wh(x,e)  denote  the  usual 

Lagrangian  of  Problem  P, (e)  and  let  K(x,0)  denote  the  set  of  Kuhn- 

4 

Tucker  vectors  (u,w)  corresponding  to  a  solution  x  of  P^(e)  at  e  =  0, 
let  Q(z)  =  (f*(Sz)  -  f*(0) )/b,  wnere  zeE^  is  a  unit  vector.  As 
above,  R(e)  denotes  the  feasible  region,  S(e)  the  solution  set,  and 
the  functions  f,  g,  h  are  assumed  jointly  once  continuously  differen¬ 
tiable  in  (x,e).  The  following  results  hold  [44]  for  Problem  P^(e). 
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Theorem  3.6.  If  R(0)  4  0,  R(e)  is  uniformly  compact  near  c  =  0,  and 

k 

CQ1  holds  for  some  x*eS(0)  then,  for  any  zeE  , 


lim  inf  Q(z)  >_  min 


3-ht 


(u,w)eK(x*,0) 


zV  L(x*,u,w,0) 


(3.1) 


'The  next  corollary,  following  immediately  from  the  theorem,  gives 
a  result  that  is  weaker,  but  useful  in  the  sequel. 

Corollary  3.7.  If  R(0)  4  R(e)  is  uniformly  compact  near  e  =  0, 
and  CQ1  holds  at  each  xeS(O)  then,  for  any  zeE  , 

lim  inf  Q(z)  inf  min  zV  L(x,u,w,0)  (3.2) 

3->0+  xeS(O)  (u,w)eK(x,0) 


Theorem  3.8.  Under  the  conditions  of  Corollary  3.7,  for  any 

„k 
zeE  , 


lim  sup  Q(z)  <  inf 


max 


3-HT 


xeS(O)  (u,w)eK(x,0) 


zVt.L(x,u,w,0)  (3>3) 


Corollary  3.9.  If,  in  the  hypotheses  of  Corollary  3.7,  CQ1  is  replaced 
with  CQ3,  then  for  each  zeE  ,  D  f*(0)  exists  and 


D  f*(0)  =  inf  zVL(x,u(x) ,w(x) ,0) 
z  xeS(O) 


(3.4) 


where  (u(x),w(x))  is  the  unique  optimal  Lagrange  multiplier  vector 
associated  with  xeS(O) . 

The  following  theorem  obtained  by  Fiacco  and  Hutzler  [44]  corres¬ 
ponds,  under  slightly  different  assumptions,  to  results  obtained  by 
Gol' stein  [57]  and  Hogan  [66],  for  a  general  class  of  problems  that 
are  convex  in  x. 


Corollary  3.10.  Let  -f  and  g^  ,  1  =  l,...,m,  be  convex  functions 
in  x,  and  let  the  functions  h.,  j  =  l,...p,  be  affine  in  x,  with 

1  3 

all  functions  jointly  C  in  (x,c).  If  R(0)  4  0,  R(e)  is  uniformly 

compact  near  t  =  0 ,  and  CQ1  is  satisfied  for  each  xeS(O),  then  D  f*(0) 

k  Z 

exists  for  each  zeE  ,  and 


D  f*(0)  =  inf  max 


xeS(O)  (u,w)eK(x,0) 


zV^  L(x,u,w,0  ) 


(3.5) 
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Lerapio  and  Maurer  [77]  have  obtained  similar  bounds  under  analogous 
assumptions  that  are  required  to  handle  general  perturbed  infinite  dimen¬ 
sional  programs  of  the  form  minimize  f(x,c),  subject  to  xeK^  and  gix,c)cR2> 
where  and  R^  are  arbitrary  closed  convex  sets.  Auslender  [10]  has 
also  obtained  these  bounds  for  the  right-hand  side  pertubation 
problem  P.^(e),  extending  the  results  of  Gauvin  and  Tolle  [50]  by  using 
a  weaker  form  of  the  Mangasarian-Fromovitz  constraint  qualification. 

This  allows  him  to  replace  the  differentiability  assumption  on  the 
objective  and  inequality  functions  with  the  weaker  requirement  that 
t  iey  be  locally  Lipschitz. 

Although  we  are  focusing  attention  on  programs  for  which  the 
spaces  involved  are  finite  dimensional,  we  note  that  most  of  these 
sensitivity  results  have  been  extended  to  infinite  dimensional  programs. 

For  example,  Maurer  [82,83]  has  recently  obtained  a  characterization 
of  the  directional  derivative  of  the  extremal  value  function  subgradient 
for  problem  P^(e),  and  lias  applied  his  results  to  a  class  of  optimal 
control  problems. 

3. 3  Lipschitz  Properties 

In  t.iis  section,  we  will  consider  problem  P^(e).  Following  the 

notation  used  in  Section  2,  recall  that  R(e)  =  {xtE  :  g(x)  >_  e}, 
k. 

B  =  {e  in  E  :  R(e)  ^  0),  and  f*(c)  =  inf  {f(x):  xeR(e)}.  In  addition, 
let  S(e,6)  =  (xeR(e):f(x)  <_  f*(t)  +  6  for  6  >  0). 

Stern  and  Topkis  [111],  defining  a  notion  of  linear  continuity, 
establish  conditions  under  which  f*(e)  satisfies  a  Lipschitz  condition. 
Under  convexity  assumptions  on  the  problem  functions,  they  also  show 
that  S(e,6),  the  set  of  6-optimal  solutions,  is  continuous. 

Definition  3.11.  The  real-valued  function  f(x)  is  said  to  satisfy  a 
Lipschitz  condition  on  a  set  S  if  there  exists  a  value  M  >  0  such  that 
|f(x)-f(y)|  <_  M  •  ||  x-y  ||,  for  all  x,  yeS. 
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Def  inition  I.  12.  Suppose  ^  is  a  point -to-set  mapping  1  rom  VCE™  to  sub¬ 
sets  of  E°.  Then  <js  is  said  to  be  uniformly  linearly  continuous  on  YC  Y 
if  there  exists  a  value  M  >  0  such  that  inf  JJz-x||  M-||y-y|j,  for  all 

xe<Ky) 


ze<)>(y)  and  for  all  y,ycY. 


Theorem  3.13.  Let  BCB  and  suppose  R(e)  is  bounded  for  c  in  B.  If  R(e) 
is  uniformly  linearly  continuous  with  constant  K  on  _<  e},  and 

if  f  satisfies  a  Lipschitz  condition  with  constant  M  on  R(c),  then  f*(e) 
satisfies  a  Lipschitz  condition  with  constant  KM  on  Bflfeie  _<  e}. 


Theorem  3.14.  If  f,  and  -g  ,  i  =  are  strictly  quasi-convex, 

R(c)  is  bounded,  and  1(e)  4  0,  then  S(e,6)  is  continuous  at  e  for  each 

6  >  0. 


Hager  [61]  has  recently  obtained  Lipschitz  results  for  quadratic  programs 
with  unique  solutions.  Hager  obtained  the  Lipschitz  continuity  of 
the  solution  of  such  programs  under  the  hypothesis  that  the  gradients 
of  the  binding  constraints  satisfy  an  independence  criterion.  Under 
these  same  conditions,  the  Lipschitz  continuity  of  the  (unique) 

Lagrange  multiplier  vector  is  also  obtained.  In  both  instances, 

Hager  provides  an  estimate  of  the  Lipschitz  constant. 
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4 .  I mp 1 i c i t  Function  Theorems 

There  are  many  forms  of  implicit  function  theorems  which  have  found 
extensive  application  in  functional  analysis.  These  theorems  treat 
the  general  problem  of  solving  an  equation  of  the  form 

f  (x,y)  =  z  (4.1) 

for  x  in  terms  of  y  and  z.  The  classical  results  in  this  area  are  by 
now  well  known.  For  completeness,  we  present  two  forms  of  the  implicit 
function  theorem.  A  more  complete  discussion  of  these  theorems  is 
contained  in  [16], 

n+m  n 

Theorem  4.1.  Suppose  f:  E  -*■  E  is  a  continuously  differentiable  mapping 
whose  domain  is  T.  Suppose  (x,y)  eT,  f(x,y)  =  0,  and  f'(x,0)  =  0  implies 
x  =  0.  Then  there  exists  a  neighborhood  of  y,  N(y) C£m,  and  a  unique 
function  ge  c'(N(y)),  g:N(y)-*En,  with  g(y)  =  x  and  f('g(y),y)  =  0  for  all 
yeN(y) . 


The  function  g  is  said  to  be  defined  implicitly  by  the  equation 
f(g(y) ,y)  =  0.  In  the  next  theorem,  the  notation  f.  is  used  to  denote 
the  partial  derivative  of  f  with  respect  to  its  jth  argument. 


Theorem  4.2.  If  f  (x^, . .  .  .x^jy^  , . . .  ,y  p  is  analytic  in  a  neighborhood  of 

the  origin  for  j  =  l,...,k,  with  f(0,0)  =  0  and  [  3  (f  j. ,  .  .  .  ,f  k  )/3  (Xj  , .  .  .x^ )  ] 

exists  at  x  =  y  =  U,  then  the  system  of  equations  f .  (x.  , .  . .  ,x.  jy..  , . .  .y  )  =  0, 

j  i  k  i  x 

for  j  =  l,...,k,  has  a  unique  solution  x.  =  x . (y  , . .  . ,y . ) ,  wnich  vanishes 

J  J  t  i 

for  y  =  0  and  which  is  analytic  in  a  neighborhood  of  the  origin. 


Results  of  this  type  have  particular  applicability  to  sensitivity 
analysis  in  nonlinear  optimization  and  have  only  recently  been 
exploited.  Hildebrandt  and  Graves  [63]  have  provided  results  on  the 
existence  and  differentiability  of  solutions  of  equation  (4.1).  Cesari 
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l 20)  has  established  conditions  under  which  the  equation  f(y,y)  =  0  has 
at  least  one  solution,  and  discusses  the  continuous  dependence  of  y 
on  parameters  ol  the  equation.  Rheinboldt  [93)  lias  given  global 
existence  theorems  for  the  solution  of  (4.1)  which  leads  to  a  "continu¬ 
ation  property".  This  continuation  property  has  been  applied  to  the 
solution  of  parametric  optimization  programs. 

Fiacco  and  McCormick  [45]  provided  a  first  application  of  an 
implicit  function  theorem  to  obtaining  sensitivity  information  about 
the  solution  of  a  mathematical  program.  Since  then,  additional  results 
in  this  area  have  been  obtained  by  Bigelow  and  Shapiro  [14],  Armacost 
and  Fiacco  [3, 4, 5, 6],  Armacost  [1],  Fiacco  [41],  and  Robinson  [98]. 

In  [94],  Robinson  provided  a  implicit  function  theorem  for 
variational  inequalities  of  the  form 

0  e  f(x,e)  +  T ( x) ,  (4.2) 

where  f :  XxP  E  ,  X  Is  an  open  set  in  e'1  and  P  is  a  topological  space, 

and  T:E  — •  En  is  a  maximal  monitone  operator.  Robinson  showed  that 

If  f  is  continuously  (Predict)  differentiable  on  XxP,  and  X  CX  is 

o  — 

nonempty  and  bounded,  then  the  set  of  solutions  S(e)  of  (4.2)  is 
u.s.c.  in  a  neighborhood  of  e  =  t  ;  S (e)  =  X^;  and  for  each  S  >  0, 

0  +  S(e)  C  S(e)  +  (X  +  5)  a (e)  B,  (4.3) 

where 

a(c)  =  max  { (  j  f ( x , e )  -  f(x,0)|j:  xeXQ}, 

B  is  the  unit  sphere  in  En,  and  X  is  a  Lipschitz  modulus  regulating 

(Lf  +  T)_1.  Here, 
o 

Lf  (♦)  =  f(x  ,e  )  +  F(x  ,e)  [(•)  -  x  ] 
o  o  o  o 

where  F,  the  Frechet  derivative  of  f  ,  is  positive  semi-definite  and 

(Lf  +  T)-1  (0)  =  X  . 
o  o 

As  Robinson  indicates,  nonlinear  complementarity  problems,  and  thus 
the  Kuhn-Tucker  conditions  for  mathematical  programming,  can  be  written 
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in  the  form  of  (4.2).  Robinson's  result  (see  l 95])  when  applied  to 
linear  equations  of  the  form 

0  e  Ax  +  a  +  T(x) ,  (4.4) 

where  A  is  an  n  x  n  matrix,  ae£n,  and  T(x)  is  the  subdifferential  opera¬ 
tor  of  the  indicator  function  of  a  nonempty  polyhedral  convex  set, 
leads  to  the  relation 

0  t  S(A'  ,a'  )  fl  X  C  S  (A,a)  +  Ac  '  (1-Ae  '  f'(l+u)  B.  (4.5) 

Here  S(A,a)  is  the  set  of  solutions  of  (4.4),  X  is  any  bounded  open 
set  containing  S(A,a),  e'  =  max  { ] | A ' —A ) | ,  ||a'-a||},  p  is  a  bound  on 
S(A,a)  and  A  is  a  Lipschitz  modulus  for  [A  +  a  +  T]  ^ . 

These  results  can  be  applied  directly  to  quadratic  programs,  and  in 
that  context  (4.5)  can  be  viewed  as  an  extension  of  Daniel's  [22]  result 
on  the  solution  stability  for  definite  quadratic  programs,  which 
although  it  does  not  involve  an  implicit  function  theorem,  is  given 
next  for  comparison  with  (4.5).  Daniel  considers  the  program 
min  *£  (x'Kx)  -  xk 

s  .  t.  Gx  _<  g 

Dx  =  d, 

where  K  is  positive  definite  and  symmetric  with  A>0  its  smallest 
eigenvalue.  Letting  j  |  •  |  |  be  the  St^  norm,  Daniel  obtained  the  following 
special  case  of  (4.5). 

Theorem  4. 3.  If  e  =  max{ j | K' -K | | , | | k ' ~k  j | } ,  then  for  e  >  A , 

I IXq  “  XQI I  1  e  1(1  +  I  U0I I), 

where  xq  solves  the  program  above  and  x^  solves  that  program  when  K' 
and  k'  replace  K  and  k,  respectively. 
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5 .  First-  and  Second-Order  Sensitivity  Analysis 

Using  additional  assumptions,  a  number  of  stronger  results  have 
been  obtained  which  characterize  more  completely  the  relationship 
between  a  solution  set  and  the  optimal  value  function  of  a  mathema¬ 
tical  program  to  general  perturbations  appearing  simultaneously 
in  the  objective  function  and  anywhere  in  the  constraints.  These 
problems  generally  have  the  form  1^(0,  which  we  treated  briefly 
in  Section  3  and  we  formulated  as  follows: 


minimize  f(x,c) 

s.t.  g(x, e)  _>  0  I’4(c) 

h(x,e)  =  0, 


,.P 


where:  f:E  xE  +E  ,  g:E  x  E" 

Fiacco  and  McCormick  [  45]  have  obtained  conditions  which 
guarantee  the  existence  of  a  differentiable  function  of  e  which 
locally  solves  a  particular  form  of  P4(s).  Fiacco  [41],  Armacost 
and  Fiacco  [3, 4, 5, 6],  and  Robinson  [98]  have  extended  this  result 
to  programs  in  which  the  pertubations  appear  as  in  P^(e).  All  of 
these  results  rely  on  a  form  of  the  implicit  function  theorem  in 
order  to  establish  the  existence  of  a  differentiable  solution  of 
P^(c).  The  next  theorem,  due  to  Fiacco  [41],  establishes  the 
existence  of  a  once  continuously  differentiable  (local)  solution 
of  P4(e). 


Theorem  5.  1 . 

FI) 

F2) 


F3) 

F4) 


If 

2 

g,  h  are  C  in  (x,c)  in  a  neighborhood  of  (x*,0), 
the  second-order  sufficiency  conditions  hold  at 
[x*,u*,w*] , 

the  linear  independence  (Li)  assumption  holds  at  x*,  and 
* 

u.  >  U  for  all  i  such  that  g.  (x*)  =  0,  i.e.,  strict 

1  X  jju 

complementary  slackness  (SCS)  with  respect  to  u  holds 
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i)  x*  is  a  local  isolated  minimizing  point  of  P  (0)  with 

4 

unique  Lagrange  multipliers  u*,  w*, 

ii)  for  e  near  0,  there  exists  a  unique  function 

y(e)  =  [x(e) ,u(e) ,w(e) ]  satisfying  the  second-order 
sufficiency  conditions  for  problem  l’^(c),  with 
y(0)  =  [x*,u*,w*] ,  and 

iii)  for  e  near  0,  the  gradients  of  the  binding  constraints 
are  linearly  independent,  and  strict  complementary 
slackness  holds  for  u(e)  and  g(x,e). 


Jittorntrum  [ 70]  has  obtained  results  that  do  not  require  the 
strict  complementarity  condition  (F4).  Under  these  weakened  assump¬ 
tions,  (i)  and  (ii)  (less  the  differentiability  of  y(t))  pertain, 
along  with  the  local  differentiability  of  f*(t)  =f  (x(e),e)  at  e  =  0. 
In  addition,  the  triple  (x(e) ,u (e) ,w(c) )  is  shown  to  satisfy  a 
Lipschitz  condition  in  each  of  its  components  at  e  =  0. 

Spingarn  [110]  has  also  extended  the  above  result  by  considering 

2 

the  problem  P,  (e)  with  additional  constraints  that  restrict  e  to  a  C 
submanifold,  P,  in  E  and  restrict  x  to  a  " cyrtohedron"  of  class 
C2  in  En.  He  has  shown  a  certain  set  of  second-order  conditions  to 
be  necessary  for  optimality,  and  that  these  conditions  also  imply 
the  results  obtained  by  Fiacco  [41].  Before  stating  this  result, 
we  must  introduce  adult ional  notation  and  define  several  terms. 

This  notation  follows  that  found  in  [110]. 


jeB, 


Let  A  and  B  be  finite  (possibly  empty)  index 
let  {g^ }  and  (tu  }  be  finite  collections  of 


sets,  and  for  icA  and 
functions  defined 


and  the  open  set  UCE.  Also,  for  xeU  and  A'CA,  let 


r(x,A')  =  { Vg . (x) :  isA ' }  U  {Vh. (x)  :jeB) ,  and 

J 

Z(a')  =  (xeU:g^(x)  =  0  =  h.(x)  for  all  icA'  and  jcBl. 
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Definition  5.2.  Dot  S  be  a  nonempty  connected  subset  of  En.  Then, 

K.  k 

for  K-Jl,  S  is  a  cyrtohedron  of  class  C  if  there  exist  sets  of  C 
functions  {g^l  for  ieA  and  {h^}  for  jtB,  defined  on  a  neighborhood  N 
of  x*L.£n  with: 

i)  x*eZ(A),  and  for  xcN,  xeS  if  and  only  if  g.(x)  >  0 

for  all  icA  and  h^ (x)  =  0  for  all  jeB, 

ii)  if  -5~a .  Vg .  +y^b,Vh.  =  0  for  some  a.,  b.  with  a  >  0,  then 

^  l  *->  j  i  i  J,  i~ 

a.  3  b.  =0  for  all  i,  i,  and 
i  J 

iii)  ACA^CA  and  1  (x* ,A^)  c  span  T(x*,A  )  implies  that 
Z(Aq)  =  Z(AX). 

Consider  now  the  problem  pj  (e)  which  is  P. (c)  with  the  additional 
constraints  mentioned  above.  The  following  definition  contains  condi¬ 
tions  which  are  sufficient  for  optimality  in  p^(c). 

2 

Definition  5.3.  Let  S  be  a  cyrtohedron  of  class  C  .  The  point  y*  = 
(x*,u*,w*)  is  said  to  satisfy  the  strong  second-order  conditions  for 

P£(e>  if: 

i)  x*e{x:g(x,e)  >  0}fl{x:h(x,e)  =  0j, 

ii)  -  VI,(x*,u*,w*,c)  is  the  relative  interior  of  the 
normal  cone  to  S  at  x*, 

iii)  the  gradients  of  the  constraints  that  are  binding  at 
x*  are  linearly  independent, 

iv)  for  each  i  =  l,...,m,  u*  >  0  if  and  only  if  g  (x*,e)  =  0, 
and 

v)  z' [V^L(x*,u*,w*,e)  +  K(VL(x* ,u* ,w* , e) ) ] z  >  0  for  all 
non-zero  z  e  En  for  which 

a)  z  is  in  the  largest  linear  subspace  contained  in 
the  tangent  cone  to  S  at  x«, 

b)  z'Vg.(x*,e)  =  0  for  all  ieB*(£),  and 

c)  z'Vh.(x*,c)  =  0  for  j  =  l,...p. 
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where  K(*),  the  curvature  of  the  facial  submanifold  of  S  which 
contains  x*,  is  an  n  x  n  matrix. 

If  the  set  S  is  taken  to  be  En,  then  P'(c)  reduces  to  the  program 
P^(c)  and  (ii)  and  (v)  above  become  the  familiar  conditions 

ii')  VL(x*,u*,w*, e)  =  0,  and 

v')  z'V2L(x*,u*,w*,e)  z  >  0  for  all  non-zero  zeEn  for  which 
(b)  and  (c)  above  hold. 

With  this  background,  we  now  state  Spingarn's  result. 

Theorem  5,4.  Consider  the  problem  P ^ (e) .  If  the  strong  second-order 

conditions  hold  at  y*=(x*,u,v,w*)  eS  x  E™  x  E^,  £*  in  E  ,  then  there 

Ic  n 

exist  neighborhoods  NCE  and  Xc  E  of  c*  and  x*  respectively,  and  a 
function  y(e)  =  (x(e) ,u(e) ,w(e))  defined  on  N  such  that: 

i)  y(e)  satisfies  the  strong  second-order  conditions  for  P^(e), 

ii)  for  each  z  in  N,  xC^is  an  isolated  local  minimizer 

for  P' (e) ,  and 
4 

iii)  for  each  z  in  N,  the  Lagrange  multipliers  u(c),  w(e), 
associated  with  x(e)  are  uniquely  determined. 

Under  slightly  weaker  assumptions  than  those  invoked  by  Fiacco 
(41],  Robinson  [98 |  has  obtained  results  similar  to  those  stated  in 
Theorem  5.1,  proving  the  continuity  of  the  K.uhn-Tucker  triple,  and  using 
the  results  to  derive  bounds  on  the  variation  of  y(e). 

Theorem  5,5.  Let  B  be  a  Banach  space,  S  C  B,  X  C  E  ,  witli  X  and  S  open  sets. 
Let  f,  g,  and  h  have  second  partial  derivatives  with  respect  to  x  which 
are  jointly  continuous  on  X  x  S.  For  e*  in  S,  suppose  (x*,u*,w*)  is  a 
Xuhn -Tucker  triple  of  P4(0-  Also  assume  that  LI,  SCS,  and  the 
second-order  sufficiency  conditions  apply  at  ( x5' , u", w" )  .  Then 
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i)  there  exists  a  continuous  function  y(c)  with 

y(c“)  =  (x*,u*,w*),  and  for  each  e  in  S,  y(e)  is  the  unique 

Kuhn-Tucker  triple  of  'i^(e)  and  the  unique  zero  of 

|VL(x,u,w,e),  u  g  (x,t) . u  g  (x,c), h, (x,c) , . . . ,h  (x,t)], 

i  m  m  L  P 

ii)  for  e  near  c*  x(e)  is  an  isolated  local  minimizing 

point  of  l>4(c)  ,  and 

iii)  LI,  SCS ,  and  the  second-order  sufficiency  conditions 
hold  for  e  near  e*. 

Theorem  5.6.  Linder  the  hypotheses  of  the  previous  theorem,  for  any 

f  ^  2 

Ae(0,1),  there  exist  neighborhoods  N.  of  e  and  N.  of  (x*,u*,w*.)  such  that 
1  2  A  A 

for  any  e  in  N  and  any  y  in  N.  we  have: 

A  A 

I ly  -  y(e*)  | |  1  (1  -A)"1  | |  M(y(e*),e*)_1| |  -  | |G(y,e) [ { , 
where  M  is  the  Jacobian  of  G  =  [  VL(x,u,w,e),  u^g^, . . . , u^g^,  ]• 

If  instead  of  the  Lagrangian  of  P^(e)  we  consider  the  logaritlimic- 
quadratic  penalty  function:  n  ,, 

W(x,c,r)  =  f(x,e)  -  r  ^  Sin  (g(x,e)  +  l/(2r)  h(x,c), 

we  have  the  following  theorem  due  to  Fiacco  134]. 

Theorem  5.  7.  Assume  (F1)-(F4)  above.  Then,  in  a  neighborhood  of 
(e,r)  =  (0,0)  there  exists  a  unique,  once  continuously  differentiable 
function  y(e,r)  =  [x( e,r) ,u( e,r) ,w( e,r) ]  satisfying: 

(KT1 )  VL(x,u,w,S)  =  0 

(KT2)  u^g^(x,e)  =  r  i  =  and 

(KT3)  h.(x,e)  =  w  _ r  j  = 

J,  _  J 

with  y(0,0)  =  [ x* , u* , w* ] .  Furthermore,  for  any  (e,r)  near  (0,0)  with 
r  >  0,  x(e,r)  is  a  locally  unique  unconstrained  local  minimizing  point 
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o 

of  W ( x , £ , r )  with  g  (x(c,r),e)  >  0  for  each  i  =  and  V  W(x(t,r),  ) 

positive  definite. 

Using  a  number  of  the  results  stated  above,  Fiacco  141]  has  obtained 
first-order  (Taylor)  approximations  of  y(e).  The  existence  of  higher 
order  derivatives  of  y(e)  is  easily  seen  to  depend  on  the  degree  of 
(continuous)  differentiability  of  the  problem  functions.  This  follows 
directly  from  an  application  of  the  implicit  function  theorem  (cf.  [13]). 

An  analogous  result  holds  for  y(e,r).  In  fact,  Fiacco  [41]  has  shown 
that,  under  the  appropriate  conditions,  not  only  do  higher  order 
derivatives  of  y(e,r)  exist,  but  these  derivatives  converge  to  the 
corresponding  derivatives  of  y(e). 

Theorem  5. 8.  Let  f,  g,  and  h  have  continuous  derivatives  of  all  orders 
up  to  k+1.  Assume  that  (F2)-(F4)  apply.  Then,  in  a  neighborhood  of 
(e,r)  =  (0,0),  there  exists  a  unique  function  y(e,r)e  C  , 
y(e,r)  =  [x(e, r) ,u(e , r) ,w(e , r) ]  satisfying  (KT1)-(KT3),  with 

y(e,r)+  y(e) ,  and 

[V /de^  ]y  (e  ,r)->[d'’  I  dz^  ]y  (e)  j  =  l,...,k, 
as  r-*-  0  for  (e,r)  near  (0,0). 

Armacost  and  Fiacco  [2]  have  illustrated  computational  aspects  of 
the  convergence  properties  outlined  above.  Using  the  SUMT  (Sequential 
Unconstrained  Minimization  Technique)  computer  code  developed  by 
Mylander,  Holmes,  and  McCormick  [91],  and  a  subroutine  for  sensitivity 
analysis  coded  by  Armacost  and  Mylander  [7]  that  implements  a  procedure 
based  on  the  results  given  in  Theorems  5.12  and  5.13,  they  demonstrated 
the  convergence  of  the  first  partial  derivatives  of  the  optimal  solution  and 
the  optimal  value  function  of  several  problems.  Subsequently,  Armacost 
and  Fiacco  [6]  used  this  computer  program  to  analyze  the  behavior  of  the 
solution  of  an  inventory  problem  relative  to  changes  in  several  problem 
parameters . 

Armacost  and  Fiacco  [6]  have  established  the  first-  and  second-order 
changes  in  the  optimal  objective  function  of  the  problem  '’^(t)  by  way  of 
the  following  theorem. 
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Theorem  5^9.  Assume  that  (F1)-(F4)  hold  for  P^(c).  Then,  in  a  neighbor¬ 
hood  of  £  =  0,  the  extremal  value  function  f*(c)  is  twice  continuously 
differentiable  as  a  function  of  e,  and 

i)  f*(e)  =  L*(e) , 

ii)  Vef*(e)  =  VeL(x(e) ,u(e) ,w(e) ,e) ,  and 

iii)  V2f*(e)  =  V^[ V^L(x(e) ,u(e) ,w(e) ,e)  ]. 

This  sensitivity  result,  obtained  for  the  usual  Lagrangian,  has 

been  extended  by  Armacost  and  Fiacco  [3]  to  the  augmented  Lagrangian 

used  by  Buys  [18]  and  Buys  and  Gonin  119]  to  obtain  related  sensitivity 

results.  The  augmented  Lagrangian  is  defined  as: 

<j>(x,u,w, e,c)  =  f (x,e)  -£T(u  -  (l/2)cg  (x,e))g(x,e) 

p  ieJ  i  i  „ 

+  zl(w.+  ( l/2)ch  .  (x,e) )h  .  (x,e)  -  (l/2c)y'  u.  , 

j-1  J  J  J  Uk  1 

where  J  =  {i:u  -  cg(x,0)  >0}  and  K  =  [i:u  -  cg(x,0)<  0}. 
i  i  i  i 

Theorem  5. IQ.  Under  the  assumptions  (F1)-(F4),  for  £  near  0  and  c  >  c*, 
there  exists  a  unique  C1  function  y(e,c)  =  [x(e,c) ,  u(e-,c)  ,w(e  f  c) ,  u(e,c)  ] 
satisfying : 

i)  V<}>(x,u,w,e  ,c)  =  0 

ii)  u^g(x,e)  =0  i  =  l,...,m,  and 

iii)  h^x,e)  =0  j  =  1 , . . . ,p, 

with  [x(e ,c) ,u(e ,c) ,w(e,c) ]  =  y(e).  Furthermore,  for  any  e  near  0  and 
c  >  c*  we  have  that  x(e,c)  is  a  locally  unique  unconstrained  local 
minimizing  point  of  <j>  [x,u(e  ,c)  ,w(e  ,c)  ,e  ,c]  and  V24>  is  positive 
definite  for  [x,u,w]  near  [x* ,u*,w*] . 

Armacost  and  Fiacco  [5]  have  also  obtained  first-  and  second-order 
expressions  for  changes  in  the  extremal  value  function  as  a  function 
of  right  hand  side  perturbations.  Consider  the  problem 

minimize  f(x) 

s.t.  g(x)  >  e.  1  =  l,...,m  P  t (c) 

1  J 

h(x)  =  e  3  =  !,•••  ,P 

j+m 
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The  Lagrangian  for  ^(r)  is  given  by: 

L ( x , u , w , e )  =  f(x)  -£  u  | g ( x )  -e  |  + 
i=l  i  i  i 

The  following  theorem  applies  to  this  cons 

Theorem  5J.1.  Let  f,  g,  and  h  be  twice  continuously  differentiable  in  x 

in  a  neighborhood  of  x*  for  e  near  0.  Assume  (F2)-(F4)  apply  to  p^(e), 

Then,  in  a  neighborhood  of  e  =  0, 

i)  V£  f*(e)  =  [u(e),-w(e)]  ,  and 

ii)  t  f*(e)  =  I  V  u(e) ,-V  w(e) ] . 
e  e  e 


Z  W  |  ll  ( x )  -  F.  ] 

j=l  j  j  j+m 

t  ruct . 
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6.  Additional  Results,  Applications  and _F ut ur e  Res e arch  , 

We  have  endeavored  to  present  a  number  of  basic  contributions  to  the 
theory  of  sensitivity  and  stability  analysis  for  general  classes  of  nonlinear 
programming  problems.  Hopefully,  we  have  captured  the  main  thrust  and  range 
of  developments  for  the  static  nonconvex  deterministic  problem  that 
relate  the  behavior  of  the  optimal  value  or  solution  set  to  perturbations 
of  problem  parameters.  We  have  in  this  brief  account  omitted  a  number  of 
interesting  results  and  implications  that  weave  the  fabric  more  tightly, 
and  we  nave  not  covered  results  that  significantly  exploit  additional 
problem  structure.  In  particular,  there  are  numerous  results  obtained 
by  Rockafellar  in  his  book  [104]  and  in  several  papers  that  utilize 
convexity  and  duality  properties  to  characterize  stability.  Geoffrion 
[52 j,  Gol'stein  [57]  ,  Hogan  [68]  and  others  have  also  contributed  further 
basic  results  in  convex  programming .  Quadratic  programming  stability 
characterizations  have  been  rather  thoroughly  developed  by  Boot  [17], 

Cuddat  [60],  and  Daniel  [22  1,  while  the  parametric  range  analysis  techniques 

for  linear  programming  are  now  well-known  and  routinely  implemented,  with 
the  stability  characterizations  being  further  extended  by  Mills  [88], 

Williams  [113],  Dantzig,  Folkman  and  Shapiro  [25],  Bereanu  [12],  Martin 
[81],  and  Robinson  [95,100].  Further  results  involving  the  exploitation 
of  other  structures,  for  example  by  Dembo  [26]  for  geometric  programming, 

'nave  recently  been  obtained,  and  others,  e.g.,  for  separable  136]  and 
factorable  programming  [83]  should  be  forthcoming. 

It  should  also  be  emphasized  that  more  general  treatments  of  parametric 
stability  results  have  been  obtained  recently,  primarily  by  Robinson 
[94-101],  who  has  provided  both  a  basic  theoretical  framework  and  a 
number  of  deep  stability  characterizations.  Their  generality  encompasses 
applications  to  complimentarity  and  equilibrium  problems,  as  well  as  to 
mathematical  programming  problems,  both  in  finite  and  infinite  dimensional 
spaces,  and  involve  effective  use  of  monotone  operators,  convex  analysis 
and  contemporary  techniques.  We  should  also  mention  the  many  general 
results  obtained  by  Rummer  [74]  which  synthesize  a  good  bit  of  the  theory 
utilizing  the  point-to-set  mapping  constructs. 
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We  should  also  mention  that  an  entire  body  of  theory,  somewhat  more 
general  but  to  a  great  extent  anologous  to  the  theory  involving  parametric 
variation,  has  developed  more  or  less  in  parallel  with  the  parametric 
theory.  This  involves  the  study  of  the  effects  of  general  perturbations 
of  the  problem  functions  on  solution  behavior.  For  example,  consider 

min^f  (x)  s.l.  g  (x)  >_  0,  li  (x)  =  0,  where  the  functions  fk,  gk,  h^ 
converge  in  some  specified  sense  to  f,  g,  h,  respectively,  as  k  -*■  °o. 
Questions  concerning  the  relationship  of  solutions  of  to  solutions  of 
P.  rain  f (x)  h.L.b(x)^0,  h(x)  =  0,  are  oi  interest  and  obviously  relate 
to  sensitivity  and  stability  questions.  Numerous  references  could  be 
given  here,  in  addition  to  those  already  provided  for  the  parametric 
perturbations  which  are  certainly  relevant  (noting  that  the  problem 

P(e)  may  be  analyzed  at  e.  =  :  by  considering  problems  of  the  form  P  where 

k  k 

f  (x)  =  f(x,t^),  etc.,  where  >  ->  e  as  k  -+  ®) .  The  interested  reader 

is  referred  to  the  recent  work  of  Salinetti  and  Wets  [106 J  that  gives  a 

number  of  interesting  results  involving  sequences  of  convex  sets  and 

their  application  to  convex  stochastic  programming  as  well  as  many 

references  to  other  work  in  this  area.  This  application  reminds  us 

that  the  general  area  of  stochastic  programming  has  not  been  addressed 

in  this  survey,  either,  although  the  inevitable  presence  of  uncertainty, 

e.g.  parameters  that  are  random  variables,  would  obviously  suggest 

that  perturbation  analysis  results  characterizing  solution  sensitivity 

or  stability  would  clearly  be  applicable.  Explicit  connections  have 

already  been  made,  as  suggested  by  Salinetti  and  Wets  [106]  and  also  by 

Bereanu  J12J. 

Sensitivity  and  stability  analysis  results  are  ready  for  extensive 
computational  implementation.  Experimental  results  have  unequivocally 
demonstrated  the  practical  applicability  of  various  computational  schemes 
that  can  generate  a  wealth  of  information  that  should  be  extremely  val¬ 
uable  to  users.  The  most  extensive  computations  have  apparently  been 
performed  by  Armacost  and  Fiacco  [2,6],  utilizing  a  computer  program 
now  called  "SENSUMT".  This  interfaces  a  subprogram  calculating 
sensitivity  information  with  SUMT  [91]  and  was  developed  by  Armacost 
and  My  lander  [71,  based  on  the  theory  developed  by  Fiacco  [41].  The 
approach  defined  and  validated  in  [41]  is  based  on  utilizing  the 
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information  go no  rated  by  a  solution  algorithm  to  calculate  sensitivity 
information  as  a  solution  is  approached.  The  particular  algorithm  for 
which  the  theory  is  developed  in  detail  in  [41  J  is  the  algorithm  based 
on  tlie  logarithmic-quadratic  mixed  barrier-penalty  function  algorithm 
for  the  problem  involving  inequalities  and  equalities  and  general 
parametric  perturbations.  However,  the  general  approach  may 
conceivably  be  applied  to  any  algorithm  and  should  yield  an  efficient 
procedure  for  adding  a  sensitivity  analysis  capability  to  a  nonlinear 
programming  code,  once  the  sensitivity  formulas  appropriate  to  the 
general  algorithmic  manipulations  and  data  organizations  have  been 
obtained.  Armacost  and  Fiacco  [5]  have  obtained  efficient  formulas 
for  the  general  problem  in  terms  of  the  given  problem  functions 
(without  presupposing  a  given  solution  algorithm  is  used),  and 
have  also  developed  formulas  for  a  class  of  exact  penalty  functions 
[3),  the  latter  also  having  been  obtained  by  Buys  and  Gonin  [19]. 

Armacost  [1]  has  obtained  results  in  terms  of  an  exponential 
penalty  function  and  a  general  "sequential"  class  of  algorithms. 

Fiacco  i38[  has  provided  results  in  the  context  of  projected  gradient 
and  reduced  gradient  calculations.  Recent  computational  experiments 
using  SENSUMT  have  been  conducted  by  Fiacco  and  Ghaemi  [42,43],  who, 
for  example,  make  numerous  inferences  concerning  the  solution  of 
a  22-variable  stream-water  pollution-abatement  model  from  an  analysis 
of  the  sensitivity  information  deriving  from  64  model  parameters. 

Dinkel  and  Kochenberger  [30,31]  and  Dinkel,  Kochenberger  and  Wong  [32] 
have  also  reported  the  successful  generation  of  sensitivity 
information  for  several  geometric  programs  based  on  practical 
examples  as  well  as  the  value  of  resulting  interpretations.  Pre¬ 
liminary  computational  work  has  thus  begun  and  the  practical  feasibility 
and  applicability  of  generating  rather  intricate  sensitivity  information 
for  nontrivial  nonlinear  programming  problems  has  been  demonstrated. 
Widespread  implementation  and  routine  use  of  this  capability  is  now 
enthusiastically  encouraged  and  will  hopefully  not  be  long  in 
coming. 
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Concerning  the  recognition  by  the  scientific  community  ol 
mathematical  programmers  ot  tiie  Importance  and  practical  ability  ot 
generating  sensitivity  information  as  part  of  the  usual  output  of 
a  nonlinear  programming  code,  we  note  that  in  a  questionnaire,  "Survey 
on  Mathematical  Programming  Software  Performance  Indicators," 
circulated  by  the  Mathematical  Programming  Society  in  May  1979,  two 
items  solicit  information  regarding  the  importance  attached  to  the 
provision  of  solution  sensitivity  information.  We  are  happy  to  see 
the  emergence  of  interest  in  this  vital  requirement,  but  are  disturbed 
that  the  interest  is  so  modest  and  late  in  arriving,  even  to  sophisticated 
theoreticians  and  practitioners.  In  this  context,  we  also  note  that 
the  f i rs t  conference  to  our  knowledge  devoted  exclusively  to  sensitivity 
and  stability  questions  in  mathematical  programming  took  place  in  May 
19  79  1  401. 

Another  area  of  researcli  in  mathematical  programming  that  is 
relevant  to  sensitivity  and  stability  analysis  is  the  development  of 
bounds  on  the  optimal  value  function  value  or  on  the  components  of 
an  optimal  solution  vector.  These  bounds  are  frequently  obtained  by 
generating  simpler  functions  that  bound  the  given  problem  functions, 
e.g.,  convex  envelopes  of  the  constraint  functions,  ns  in  the 
separable  nonconvex  programming  approach  of  Falk  and  Soland  [36]  or 
convex  underestimating  or  concave  overestimating  functions,  as  in 
the  nonconvex  factorable  programming  approach  of  McCormick  [85].  But 
once  a  procedure  for  generating  simpler  "bounding  problems"  is  at 
hand,  it  can  generally  be  applied  to  a  perturbation  of  the  original 
problem  to  obtain  "simple"  (e.g.,  convex)  bounding  problems.  If 
the  perturbation  analysis  of  the  simpler  associated  problems  is 
tractable  then,  once  the  relationship  between  the  optimal  solution  of 
the  perturbed  bounding  problems  and  the  original  perturbed  problem 
is  understood,  we  have  a  procedure  for  generating  bounds  on  the 
perturbed  solution  of  the  original  problem.  Geoffrion  [  53]  gives 
a  number  of  valuable  insights  and  several  computationally  implementable 
schemes  for  obtaining  bounds  on  the  variation  of  the  optimal  value 
function  of  a  given  problem,  in  terms  of  bounds  on  the  variation  of 
the  objective  function  of  the  given  problem,  for  example.  Bounds 
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results  such  as  those  obtained  by  Daniel  122]  for  quadratic  programming, 
a  specific  instance  of  which  was  given  in  Theorem  4.3,  and  results 
at  a  very  high  level  of  generality  obtained  by  Robinson  [91]  and 
indicated  in  Section  4,  giving  bounds  on  the  optimal  solution  set  in 
terms  essentially  of  bounds  on  the  original  problem  functions,  are 
also  applicable  here.  To  our  knowledge,  there  has  been  no  theoretical 
or  computational  exploitation  of  the  approach  suggested  in  this 
paragraph,  other  than  that  reported  by  Geoff rion  [53],  Development 
of  this  idea,  including  a  study  of  the  connection  with  sensitivity 
and  stability  theory  of  other  procedures  for  generating  solution  bounds, 
for  example,  techniques  utilizing  the  use  of  interval  arithmetic 
proposed  by  Robinson  [96]  and  Mancini  and  McCormick  [78],  and  utilizing 
many  of  the  results  known  for  systems  of  equations  such  as  that 
tailored  to  the  Karush-Kuhn-Tucker  conditions  by  Robinson  [98]  and 
stated  in  Theorem  5.6,  should  be  a  suDject  of  fruitful  research. 

Some  applications  of  sensitivity  and  stability  analysis  in 
mathematical  programming  are  rather  obvious,  g.  estimation  of 
solutions  of  perturbed  problems,  given  a  solution  of  a  problem  with 
given  parameter  values,  and  determination  of  parameters  to  which  the 
optimal  value  or  solution  set  is  most  sensitive.  Most  applications 
are  reasonably  well  documented.  We  mention  several  here  for 
completeness,  along  with  some  references:  (1)  optimality  conditions 
[41,45],  (2) convergence  of  algorithms  [87],  (3)  rate  of  convergence 
of  algorithms  [98],  (4)  decomposition  [54,76],  and  (5)  implicit  function 
minimization  [28,65].  Other  applications  can  surely  be  made  to 
parametric  nonlinear  programming  and  deformation  techniques  [55,56,o^, 
90],  homotopy  continuation  methods  [3  ,47  ,Jj  ] ,  and  to  the  derivation, 
conditioning  and  acceleration  of  alogrithms  [41,45].  Conversely, 
all  of  the  areas  mentioned  can  undoubtedly  uncover  an  abundance  of 
results  that  are  applicable  to  sensitivity  and  stability  results  in 
mathematical  programming. 

The  future  should  see  a  unification  of  the  powerful  collection 
of  results  that  are  now  known  and  scattered  throughout  the  literature. 
The  reader  may  note  that  only  one  book  has  been  devoted  entirely 
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results  such  as  those  obtained  by  Daniel  [22]  for  quadratic  programming, 
a  specific  instance  of  which  was  given  in  Theorem  4.3,  and  results 
at  a  very  high  level  of  generality  obtained  by  Robinson  [91]  and 
indicated  in  Section  4,  giving  bounds  on  the  optimal  solution  set  in 
terms  essentially  of  bounds  on  the  original  problem  functions,  are 
also  applicable  here.  To  our  knowledge,  there  has  been  no  theoretical 
or  computational  exploitation  of  the  approach  suggested  in  this 
paragraph,  other  than  that  reported  by  Geoffrion  [53],  Development 
of  this  idea,  including  a  study  of  the  connection  with  sensitivity 
and  stability  theory  of  other  procedures  for  generating  solution  bounds, 
for  example,  techniques  utilizing  the  use  of  interval  arithmetic 
proposed  by  Robinson  [96]  and  Mancini  and  McCormick  [78],  and  utilizing 
many  of  the  results  known  for  systems  of  equations  such  as  that 
tailored  to  the  Karush-Kuhn-Tucker  conditions  by  Robinson  [98]  and 
stated  in  Theorem  5.6,  should  be  a  subject  of  fruitful  research. 

Some  applications  of  sensitivity  and  stability  analysis  in 
mathematical  programming  are  rather  obvious,  e.g.  estimation  of 
solutions  of  perturbed  problems,  given  a  solution  of  a  problem  with 
given  parameter  values,  and  determination  of  parameters  to  which  the 
optimal  value  or  solution  set  is  most  sensitive.  Most  applications 
are  reasonably  well  documented.  We  mention  several  here  for 
completeness,  along  with  some  references:  (1)  optimality  conditions 
[41,45],  (2) convergence  of  algorithms  [87],  (3)  rate  of  convergence 
of  algorithms  [98],  (4)  decomposition  [54,76],  and  (5)  implicit  function 
minimization  [28,65],  Other  applications  can  surely  be  made  to 
parametric  nonlinear  programming  and  deformation  techniques  [55,56,0*, 

>9  ] ,  homotopy  continuation  methods  [3'>,47  ,8j  ] ,  and  to  the  derivation, 
conditioning  and  acceleration  of  alogrithms  [41,45].  Conversely, 
all  of  the  areas  mentioned  can  undoubtedly  uncover  an  abundance  of 
results  that  are  applicable  to  sensitivity  and  stability  results  in 
mathematical  programming. 

The  future  should  see  a  unification  of  the  powerful  collection 
of  results  that  are  now  known  and  scattered  throughout  the  literature. 
The  reader  may  note  that  only  one  book  has  been  devoted  entirely 
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to  sensitivity  analysis  in  mathematical  programming,  the  book  by 
Dinkelbach  133],  and  that  appeared  in  1969  and  was  confined  to 
linear  programming.  Unification  presupposes  the  existence  of 
enough  interesting  results  to  essentially  provide  a  significant 
body  of  theory,  a  methodology,  and  this  has  only  been  extensively 
developed  in  the  recent  past.  Certain  efforts  to  synthesize  the 
theory  have  begun.  We  mention  the  works  of  Dantzig,  Folkman  and 
Shapiro  [25],  Rockafel lar  [104 ] ,  Geoffrion  [51],  Gol'stein  [57], 

Hogan  [67],  Robinson  [94-101],  Gal  [46],  Fiacco  [41],  Rummer  [74], 

Wets  [112],  Dembo  [26],  Kojima  [72],  and,  of  course  this  survey.  A  uni¬ 
fied  methodology  will  accelerate  the  understanding  of  basic  theory  and 
stimulate  algorithms  and  software  developments,  and  thus  hasten 
widespread  and  routine  implementations. 
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